Represent and Infer Human Theory of Mind for Human-Robot Interaction

This abstract is proposing a challenging problem: to infer a human’s mental state – intent and belief – from an observed RGBD video for human-robot interaction. The task is to integrate symbolic reasoning, a field wellstudied within A.I. domains, with the uncertainty native to computer vision strategies. Traditional A.I. strategies for plan inference typically rely on first-order logic and closed world assumptions which struggle to take into account the inherent uncertainty of noisy observations within a scene. Computer vision relies on patternrecognition strategies that have difficulty accounting for higher-level reasoning and abstract representation of world knowledge. By combining these two approaches in a principled way under a probabilistic programming framework, we define new computer vision tasks such as actor intent prediction and belief inference from an observed video sequence. Through inferring a human’s theory of mind, a robotic agent can automatically determine a human’s goals to collaborate with them. Our work is largely motivated by the pioneering work in Theory of Mind for a Humanoid Robot (Scassellati 2001) and a series of cognitive science studies by Baker et al. (Baker, Tenenbaum, and Saxe 2006; Baker, Saxe, and Tenenbaum 2009; Baker and Tenenbaum 2014) concerning the Bayesian Theory of Mind (ToM), which suggests an intentional agent’s behavior is based on the principle of rationality: the expectation that agents will behave rationally to efficiently achieve their goals given their beliefs about the world. Gergely et al. (Gergely et al. 1995; Gergely, H., and Kirly 2002) showed that infants can infer goals of varying complexity, again by interpreting an agent’s behaviors as rational responses to environmental constraints. Humans perform rational planning according to the present context. There is a strong support for this interpretation of causal inference being intimately related to how humans infer goals and intentions (Baker and Tenenbaum 2014; Baker, Saxe, and Tenenbaum 2009; Baker, Tenenbaum, and Saxe 2006). Inverse planning relies on the “principle of rationality” to make claims about an intentional agent’s motions and actions: if one assumes that all actions are made with the goal of efficiently completing a goal, then Copyright c © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. it is possible to infer that goal by observing the actions. Developmental psychology studies show pre-verbal infants are able to discern rational plans from unrelated sequences of actions (Gergely et al. 1995) and other kinematic properties of human actions (Gergely, H., and Kirly 2002). Understanding scenes and events is not a simple classification problem. As seen in several data sets (Schuldt, Laptev, and Caputo 2004; Laptev et al. 2008), action understanding algorithms in the field of computer vision have historically been formulated as discriminative learning problems. However, these data-driven algorithms only work for specific action categories with explicit visual patterns. We argue that actions are fundamentally interventions or responses to a dynamic world. As suggested by Pearl (Pearl 2009), agency and action may be intrinsically different from the underlying assumptions of classification at the philosophical level; a tree stump can become a chair if you sit on it. Action and agency in many cases “change” the world, and computer vision and reasoning systems can potentially benefit greatly by incorporating additional knowledge from rational planning and other traditional A.I. procedures. In this abstract, we consider stochastic inverse action planners as generative probabilistic programs (Goodman et al. 2008; Mansinghka et al. 2013) following the generative thinking of cognitive models (Goodman and Tenenbaum ; Tenenbaum et al. 2011). By applying these methods to highly uncertain computer vision tasks we hope to understand scenes and events in a more holistic way than has previously been explored. Ultimately this improved understanding is highly useful for building interactive robotics systems capable of interacting with humans on an intentional level. For example, a robot would be able to infer that a human’s goal is to move a heavy object. The robot can then provide assistance by pulling on the object from the other side.